A new Gaussian curvature of the image surface based variational model for haze or fog removal

Outdoor images are usually affected by haze which limits the visibility and reduces the contrast of the images. Removal of haze from real-world images is always a challenging task. Recently, many mathematical models have been proposed for the effective removal of haze from real-world images. However, these models may produce staircase effects or lower the image contrast or smooth the edges of the object. In this paper, we propose a model based on Gaussian curvature for the de-hazing of images. The atmospheric veil estimate is included based on dark channel prior (DCP), which can significantly reduce the artifacts on the edge of the image and increase the accuracy. The transmission map then changes to a high-quality map to reduce haze or fog from gray and color images. DCP combined with Gaussian curvature is done for the first time for image de-hazing/de-fogging. The augmented Lagrangian method is used to find the minimizer of the proposed functional, which will be a system of partial differential equations. To get fast convergence, fast Fourier transforms (FFT) is used to solve the system of PDEs. The performance of the proposed model is compared with other state-of-the-art models qualitatively and quantitatively. The proposed model is tested on various real and synthetic images which show better efficiency in staircase effects reduction, haze/fog removal, image contrast, corners, and sharp edges conservation respectively.


Introduction
The atmospheric light is dispersed in different angles due to the presence of atmospheric particles (e.g. fog, haze, smog, smoke, and mist) in the air. The incoming light blended to the layer of ambient light (air-light), which reflects atmospheric particles in the line of sight, depending on the turbidity and distance from the scene to the observer (visual range), causing low contrast, reduced visibility, distortion of colour, deterioration of essential elements of the photography scene and makes lots of trouble in the recognition and detection of a target in video surveillance system by blocking the direct scene transmission. Therefore, it is primal imperative to improve visibility and restore the essential features of the scene with a simple and effectual image restoration algorithm. The removal of haze or fog is considered to be an important procedure since haze or fogfree images are visually pleasing and can improve performance of computer vision tasks like object detection, classification, visual navigation, etc. According to Koschmieder [1], a degraded scene formed as displayed in Fig 1 can be formulated mathematically as follows where the observed hazy or foggy image intensity is I(x), the intensity of the scene is J(x), the intensity of the atmospheric light is I 1 and T(x) = e −βd(x) is the transmission map (scene reflected light captured by the signal receiver depending upon the amount of haze) corresponds to values between 0 (no visibility) and 1 (clear visibility) with degradation coefficient β and distance d(x) from scene point to the observer. The first term T(x)J(x) in the R.H.S of Eq (1) is the direct attenuation, describes the radiance of the scene while the second term I 1 (−T (x) + 1) is the air-light (result of the scattered atmospheric light due to atmospheric particles), twists the radiance of the scene. The key task to get J from an observed hazy frame I is the T estimate and I 1 atmospheric light. Many researchers have made significant progress in the estimation of transmission T using a single image [2][3][4] and took the average value from I or I as the atmospheric light I 1 as its brightest pixel. It should be noted that inaccurate assumption or estimation of atmospheric light I 1 leads towards imprecise results in the recuperation of J haze free image. Without the use of rational processes, several methods assume the pixels with dense fog to be pure white and take the transmissions in a direct relation with hazy colors [5][6][7]. Due to providing unsatisfactory transmission map they proceed towards Laplacian matting algorithm or guided filter algorithm for refinement. The inappropriate assumption of linearity between transmission map and haze colors applying by these two algorithms reflects every variance of hazy colors on transmission map without difference. Taking advantages of the physical properties of air-light map or atmospheric veil, Cho et al. [9] used a variational approach for the estimation of airlight map in order to restore a fog-free image. This approach can adequately remove the haze and satisfy the edge preserving property, but the restored images have very low contrast. For this purpose, a variety of histogram equalization methods were applied to make better the contrast of the restored images. During image de-hazing, it should be take into account that the accurate estimation of depth or medium transmission information leads towards finest dehazing results. Most de-hazing procedures are based on the pre-estimated depth or medium transmission information. Fang et al. [10] estimated the depth map and de-haze image using a total variation (TV) regularizer and [11,12] used the TV regularizer to medium transmission refining. TV regularizer has the advantage of preserving edges of the image but produces blocky effects due to piecewise constant solutions where the region is homogeneous. Total generalised variation (TGV) second order regularizer was implemented to overcome the blocking artifacts which has the ability of reducing stair-casing effects along with edge preservation [13,14].
The de-hazing methods are mainly classified in methods based on enhancement and physics. Methods based on enhancement do not take the mechanism of image degradation into account. They simply intend to make better the degraded image contrasts and ignore the colour restoration. The simple, fast and most widely used enhancement-based methods are typical retinex [15] and histogram-based method [16]. These methods may not properly work to remove visual defects of the image having inhomogeneous fog. The restored image can be restored overly and contains halo objects.
Physics-based methods design a physical model based on a hazy image's degradation phenomenon. Then, using that physical model, restore the haze-free image. In contrast of methods based on enhancement, physics-based methods yield natural, finest and accurate de-hazing results that thrust someone into attention as explained in many scientific theories and experimental results. Therefore, the main subject of this paper is the dehazing processes based on the physical model. In physics-based de-hazing methods, the dark channel prior [17] is most widely used and assuming that there are some dark pixels have a very low intensity of approximately zero in at least one of the RGB color channels in the non-sky patches of a haze free scene. Since the DCP is based on the non-sky regions, therefore it gives very poor results for images with sky regions. He et al. [17] used the soft matting algorithm to annihilate halo artifacts generated by the kernel operation and refined the transmission map. They improved the de-hazing results very well but the soft matting algorithm is very costly and time consuming. He et al. [18] also employed guided filtering in place of soft matting algorithm for transmission map optimization. The guided filtering gives similar results as that of soft matting algorithm but the de-hazing speed is very fast. The de-hazing performance of DCP is further improved by Xu et al. [19] and Li et al. [20]. Color lines in RGB space can also better approximate pixels as shown by [21][22][23]. Tarel et al. [24] assumed the atmospheric veil to be changed easily in a local medium and applied the median filter to improve the proposed de-hazing results. The median filter can't conserve the object edges well, therefore, in case of discontinuous scene depth it may fail. Meng et al. [25] used a boundary constraint and L 1 -norm-based regularization to estimate and refine the transmission map. They became successful in reducing halo artifacts but the de-hazed results have some color distortion. Nishino et al. [26] and Wang et al. [27] used Bayesian probabilistic method and Bayesian theory based on the Markov random field to calculate the scene albedo and scene depth to remove haze. By reducing the kernel operation using Bayesian-based algorithms, halo artifacts can be reduced and the problem of color distortion occurs as in Meng et al. [25]. Ancuti and Ancuti [28] employed a Laplacian pyramid and a Gaussian pyramid and proposed a multi-scale fusion-based haze removal algorithm. The algorithm have fast processing speed but the resulting image have high saturation and color distortion due to not focusing on the distribution of haze amount relative to the scene depth. Lee et al. [31] stated that only Gaussian curvature can maintain important structures (such as corners(edges) and wrinkles). The work was supported by Elsey and Esedoglu [29] and proved that the Gaussian curvature of the surface is a good preserver for important features of image as compared to mean curvature.
Many haze removal techniques have been proposed so far. The problems of halo artifacts, stair-cases like artifacts, and color distortion that need to be solved during the image dehazing/ defogging process are still unsolved. In this regard, our contributions to tackling such issues can be outlined as follows: • A new variation model based on the Gaussian curvature (GC) of the image surface of a given scene is proposed, to estimate an air-light map which preserves the important image features such as edges and wrinkles while handling the problems of halo artifacts, stair case effects and color distortions simultaneously quite well.
• We also take some prior assumptions called the dark channel prior (DCP) to approximate the transmission map. The transmission map is then changed into a depth map.
• Dark channel prior combined with Gaussian curvature is employed for the first time for image de-hazing/de-fogging.
• We implement augmented Lagrangian method (ALM) for the Gaussian curvature (GC) regularization based de-hazing model and design a special minimization procedure to minimize the augmented Lagrangian functional.
• For fast convergence, fast Fourier transform is implemented to compute the system of linear partial differential equations arisen from the minimization of the augmented Lagrangian functional.
This paper is summarized in the following. In section 2, a review of some well-known image de-hazing and defogging related methods is given. In section 3, our novel model for addressing the problem is proposed. The augmented Lagrangian method is used in Section 4 for the resolution of the Euler Lagrange PDEs system. Section 5 provides some image restoration and analysis results for outdoor and indoor scenes to demonstrate the performance of the proposed model. Lastly, we make some concluding remarks, applications and future directions in section 6. Next, we discuss some well known related image de-hazing and defogging methods as under.

Preliminaries
In this section, the following two well-known recent image de-hazing or defogging methods are reviewed. The restoration results of these have been compared with the proposed model discussed in subsection-5.4.

Edge preserving regularization based single image de-fogging model (M1)
In order to recover a fog-free scene, Cho et al. [9] (M1), based on the physical properties of air-light A, proposed a model for estimating the air-light map A. The energy functional of the model is where data fidelity term is the first term and a regularization term is the second term. W = min (I) is the image of minimal component of image I, H(�) is the Heaviside function, O is bounded open subset of R 2 , ||rA|| is the A gradient modulus, and λ is the regularization parameters that balance the influence between two terms of Eq (2). The function ψ is presumed to be an even function of class C 2 (R) as the edge preserving regularization function.
The following slightly regularized versions of the function H and δ (one-dimensional Dirac measure), denoted here by H � and d � ¼ H 0 � , are considered.
As � ! 0, they converge respectively to H and δ. The method improves the quality of the defogged image significantly and restores a fogged image in better colour. The algorithm is good even in case of heavy fog. However, without using histogram for the post-processing of the restored image, the image is not as bright as the atmospheric light, since it does not normally have the same brightness(taken by top 1% brightness pixel values) [9].

Dark channel prior based single image haze removal model (M2)
Dark Channel Prior (DCP) [4] (M2) is based on the assumption that in at least one of the nonsky space patches of a hazelnut free image there are some very low intensity dark pixels equal to zero. The J image can be formulated as following with the dark J dark channel: Since, the colour of the sky nearly approach to I 1 in hazy scenes and on removing fog or haze thoroughly may result un-natural scenes [4,17] and depth information may also be lost. TðxÞ can be computed as followŝ This process of estimating the transmission map is sensibly well. As in local patches, the transmission is taken constant which generates some blocky artifacts because the fact is that in a patch, the transmission is not always constant. Furthermore, the transmission map is refined through soft matting process.
Soft matting. He et al. [17] employed a soft matting algorithm [30] for refining the transmission map T(x) and minimized the following cost function: where T andT are the vectorial forms of T(x) andTðxÞ, λ is a regularization parameter and L is the Matting Laplacian matrix whose (i,j) element is computed as: where δ i,j is the Kronecker delta, μ k and S k are the average and co-variance matrix of the colours in windows w k , I i and I j are the colours of the scene I at pixels i and j, |w k | is the number of pixels in w k , ε is a regularization parameter and U 3 ia a 3 × 3 identity matrix. The optimal value of T is the following sparse linear system solution: where U is an identity matrix equal with L.
Recovering the scene radiance. Once the transmission map T(x) and atmospheric light I 1 is known, the scene radiance J(x) is computed from Eq (1) by: where T 0 represents a lower bound of T(x) and its optimal value is nearly 0.1. The scene radiance has low brightness as compared to atmospheric light, the restored images seem to be faint and the exposure of J(x) is increased for displaying. Approximating the atmospheric light. To improve the atmospheric light estimation using the DCP (more faster than the "brightest pixel" method), the pixels with (highest) intensity in the top 0.1% most haze opaque brightest pixels in the hazy scene I is selected as the atmospheric light I 1 . Although single image haze removal can easily enhance the appearance of images applying DCP on the degradation model but it may not work on images having scene object similar to the atmospheric light in an inherent manner and no shadow is diffused on them. The DCP underrate the transmission for these objects and some halo artifacts are also found in the resulting images. To overcome the above-mentioned drawbacks of M1 and M2 methods, a new variational model for haze or fog removal is proposed as follows:

The proposed de-hazing model (M3)
In this section, we propose a new model based on dark channel prior (a kind of statistics of the haze/fog-free outdoor scenes) and using Gaussian curvature (GC) as a regularizer. Dark channel prior depends on key perception-most local patches in haze/fog-free (outdoor) images contain a few pixels which have extremely low intensities in at least one color channel. Utilizing this prior to the haze/fog imaging model, we can straightforwardly approximate the thickness of the haze and recuperate a top-quality haze/fog-free image. This model uses advantages of both DCP and GC, which leads to good restoration results, because dark channel prior performs well for estimating the thickness of the haze/fog and recovering good quality scene, while Gaussian curvature performs well for eliminating staircase effect, preserving textures and sharp edges. The resulting model has the advantage of better preserving image regions containing textures and fine details, leading to a more natural scene while reducing the staircase effect in smooth regions.
Hence motivated by the advantages of Gaussian curvature compared to the mean curvature and total variation in 2D image de-noising pointed out by Elsey and Esedoglu [29] and Lee and Seo [31] in geometry processing and dark channel prior, here we design a Gaussian curvature of the scene surface regularization and DCP based model for single image de-hazing. Hence employing this model, reasonable improvement in image quality is obtained. The Gaussian curvature of a 3D surface S can be clearly expressed by the zero-level set function F and can be defined by where rF = (F x , F y , F z ) t is the gradient vector, jrFj ¼ ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffiffi are the Hessian matrix and its adjoint, respectively. Let us consider the surface S to be the graph of a hazy image I and the unexplored atmospheric region between the scene and sensor be the channel of air-light mapping (x, y): ! (x, y, A(x, y)). Then the relation F = −z + A(x, y) can be used to find a formula for k GC The GC of a 2D image surface may be formulated as Now, it is possible to explicate a new image de-hazing model using the following regularizer Therefore, our new designed Gaussian curvature based image de-hazing model is where the λ parameter is positive and R GC (A) is the Gaussian curvature regularization term.
To minimize the Gaussian curvature term, let us take Here, we apply some new notations where sign (�) is the sign function and (ν 1 , ν 2 ) = ν the unit normal vector) to reduce the complexity in writing the equations and use the divergence theorem when required.
In a state of proper arrangement, we drop the boundary terms.
ðÀ A xy ; A xx Þ:n ¼ 0; ðA yy ; À A yx Þ:n ¼ 0; and the Euler Lagrange equation with the above defined boundary conditions for the GC

PLOS ONE
based image de-hazing model can be written as To solve the non-linear PDE (17), an augmented Lagrangian method is employed as discussed in the next section.

Numerical implementation
Augmented Lagrangian method (ALM) plays an important role in solving constraint minimization problems and is used in several image restoration problems such as [32][33][34][35][36][37][38]. ALM convert the constraint minimization problem to unconstraint problem by incorporating the constraints in the energy functional, on account of which some additional terms arise known as Lagrange multiplier terms. In ALM, the problem is classified into different sub-problems using alternating minimization procedure that can be solved easily.
Notation. Representing a gray-scale image by an N 2 matrix and Euclidean space R N�N by V. The gradient operator (discrete) is a mapping r: where i, j = 1, . . ., N. D � þ x and D � þ y indicate forward difference operators with boundary condition (periodic)(A is periodically extended).We represent the usual inner product of V and Q as h�, �i V and h�, �i Q and the Euclidean norm of V and Q by k�k V and k�k Q that can be defined as follows: for p = (p 1 , p 2 ) 2 Q and q = (q 1 , q 2 ) 2 Q. Moreover, at each pixel (i, j), ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi is the usual Euclidean norm in R 2 . For the sake of convenience, we neglect the subscripts V and Q and simply use (�, �) and k�k to represent the usual inner product and L 2 norm. For solving the GC de-hazing model (14) with ALM, we are introducing a new dual variable q where q = rA, and come into possession of the following refined constrained optimization problem: s:t q ¼ rA: The constraint minimization problem F r f ðA; qÞ is further reformulated to get the augmented Lagrangian functional as follows: � rf GC ðA; q; mÞ ¼ HðW À AÞ þ l 2 RðqÞ þ hm; q À rAi þ where μ is the multiplier of Lagrange and r is a constant(positive). The alternating minimization procedure classify the functional into two sub-problems in order to find the optimal values of A, q and μ.

Sub-problem for q
For a given A and μ, the functional (objective) is defined by The Euler Lagrange equations for the functional Eq (20) Without applying any iterative procedures, it is very easy to solve Eqs (21) and (22) for q 1 and q 2 .

Sub-problem for A
For a q and μ given, we have The optimality condition for Eq (23) is defined by the linear PDE with the periodic boundary conditions. We implement Fourier transform (and FFT implementation) to compute the above linear equation by following [36,[39][40][41]. The Fourier transform of Eq (24) is y ÞðF ðm 2 Þ þ rF ðq 2 ÞÞ À F ðdðW À AÞÞ; ð25Þ whose solution is where F and F 0 are Fast Fourier transform and inverse Fast Fourier transform, μ k = (μ 1 , μ 2 ) and q = (q 1 , q 2 ); and Fourier transforms of operators such as D y are treated as the transforms of their corresponding convolution kernels. The iterative procedure shown in Algorithm 1 is used to solve Eq (19).

Experimental results and analysis
Some images are presented in this section to show the improved performance and efficiency of the proposed model in showing the corresponding visible edge maps, transmission maps, airlight maps, and final de-hazing results. In some scenes (outdoor and indoor) contaminated with haze or fog we describe and analyze the simulation results. The de-hazing results are then comparable to current state of the art methods and corresponding visible edge maps. For quantitative analysis, different quality descriptors/indicators (measures) are taken to assess the amount of newly visible edges, contrast, and mean visibility enhancement restoration. The entropy of the de-hazed images is also computed and compared with other recent methods. In MATLAB R2013a all of the experiments listed here and all simulations have been carried out on a Haier Win8.1 PC, Intel Core i3 CPU @ 1.70GHz with 4.00GB RAM. For better results, the values of Lagrange multipliers μ 1 and μ 2 are tuned automatically at every iteration until optimal restoration results are obtained and choose adaptively r 2 (0, 0.999) and � = β 2 (0, 1) according to the image size and type.
In this paper, four measures (indicators or descriptors) are taken to evaluate quantitatively the de-hazing outputs of the proposed model (M3) and its comparison with other recent existing methods in contrast and visibility recovery. These measures can be formulated as follows e: Let n 0 and n r be the cardinal numbers of the set of visible edges of the original I 0 image and restored image I r . Then assuming e be the rate of the restored edges in I r that can be computed by The value of e measures the amount of edges that are capable of being seen in I r and are concealed in I 0 due to haze and fog.
� r: Let � r be the geometric mean of the ratio of the gradient in the recovered image and in the original r i image relative to the visible edges for each pixel i. The � r is given by The value of � r display the essential and distinguishing attributes of the proposed model in the recovery of image contrast and forecast the average visibility enhancement of the restoration algorithm.
σ: The measure σ is obtained by normalizing the number n s of entirely black or white pixels of the algorithm after contrast restoration on the size of the image and can be formulated as where the width and height of the image is dim x and dim y . Entropy: Entropy is the corresponding states of intensity level adapted by the individual pixels in an image and is a measurement of image information content which make sense of the average precariousness of a collection of facts. It is used in the quantitative analysis, evaluation and providing better comparison of the image details. The higher entropy value result indicates more image details information.
In other words, a high entropy value means that the average amount of information in the image is high, which indicates there is a lot of information about the features or corners. Hence, the higher the entropy value of the image, the finer the contrast. The 2-D information entropy of the image reflects the average amount of information in the image, its value range is [0 8]. The larger the value is, the more uniform the gray value of the image is.  Table 1. It could be found that our approach can efficiently remove haze or fog and can restore high-quality haze-free images while generating no specious edges, halos, or artifacts. Note that the transmission map T(x) = e −βd(x) reflecting by the density of haze or fog as given in the 3rd column of Fig 2 depends on the distance d(x) from scene to the sensor and atmospherical degradation coefficient β caused by the scattering of light due to haze or fog in the atmosphere and is reputed as a grading variant of the depth map in case of both homogeneous and inhomogeneous haze or fog.

Synthetic road scene images under heterogeneous fog
This section discusses the restore image performance of the proposed model (M3), shown in Table 2 and Fig 3(a), 3(f), 3(k) and 3(p) on several synthetic road scene images under PLOS ONE

PLOS ONE
heterogeneous fog respectively. In the Fig 3, Table 2 show that the results of de-hazing can be efficiently used to treat the inland river image.

Realistic CCTV camera captured images under heterogeneous haze
In this section, the image visibility restoration performance of the proposed method (M3) was tested for images from CCTV cameras recorded under hazy weather conditions. In the Fig Table 3 show that the restoration results can be applied efficiently to defense and surveillance image processing.

Comparison with other image restoration methods
In this part, we compare the de-hazing results of our model (M3) with Cho et al. [9] (M1) and He et al. [4] (M2) algorithms. All the methods were applied to a large number of natural and synthetic data-set of hazy, foggy, and contrast-degraded images. In Figs 5 to 16, we show the comparative restore results. As shown in the de-hazing results, we can easily see that the results of our proposed model are better than other methods while maintaining edges and other image features quite well. Furthermore, we can observe also that the recovered outputs of Cho   [9] approach and the proposed approach but there is a problem of generating stair-case effects in Fig 5(e) results generated by all the three approaches. Although Cho et al. [9] approach is free of generating noise and stair-cases like artifacts in all the Figs except Fig 5(e), however, it gives very poor de-hazing results as compared to He et al. [4] and our method (M3).

PLOS ONE
fails for images containing sky regions and water (adapting shape of the sky) as shown in Fig  11((c), 11(g), 11(k) and 11(o)). Visual inspection shows that we perform more effectively with images taken at night, as shown in Fig 15 and is free of generating noise, stair-cases like artifacts and spurious edges as well. The discussion above shows how our proposed model can effectively reduce unwanted halos and our method can preserve more essential image features than other methods and can outstrip other state-of-the-art approaches in the field of image dehazing as well.

PLOS ONE
of the experimental results on MRFID images, one can analyze that the de-hazing results of the M3-model is much better than M1 and M2 models and have a little difference with clear images.

Quantitative assessment
In this part, we quantify, analyze and compare the results of a proposed model (M3) of the dehazing results with Cho et al. [9] (M1) and He et al. [4] (M2) methods given in Table 4 by  � r, σ), and entropy show the amount of corresponding newly visible edges, contrast and visibility enhancement, completely black or white pixels of the algorithms and states of intensity level adapted by the individual pixels of the restored images. By analyzing the values of indicators (e, � r, σ) and entropy post de-hazed images, we can conclude that the M3-method performs better and preserve the concealed edges of the images due to the fact of limiting contrast and visibility enhancement by removing haze and fog satisfactorily as compared to Cho et al. [9] and

Conclusion, applications and future work
This work is intended for the restoration of hazy or foggy images through the new Gaussian curvature of image surface based variation model combined with dark channel prior (DCP). The proposed method (M3) first estimates the atmospheric veil using the DCP. The transmission map is then converted to the high-quality depth map, with which the improved proposed model can be framed to obtain the haze or fog free image for both (indoor and outdoor) realistic and synthetic data set images. The proposed method can be employed for color and grey

PLOS ONE
images. The resulting augmented Lagrangian functional was efficiently solved utilizing the augmented Lagrangian method and a special minimization procedure is adopted as well. We also employ FFT to compute numerically the system of PDEs arisen from the minimization of the energy functional. We have tested several (indoor and outdoor) hazy and foggy images in order to validate the effectiveness of the proposed method. We also compared our results with other state-of-the-art de-hazing methods. The test results have demonstrated that the

PLOS ONE
M3-method is more effective while preserving sharp edges, contrast and other image details during the haze or fog removal process quite well than other comparing methods. Applications: The applications of the proposed method may be extended to cover image segmentation, image inpainting, inland river image processing, road scenes image processing under homogeneous and heterogeneous haze or fog, defense and surveillance images, underwater image processing, and hazy or foggy videos processing.
Future work: The next work involves developing the methodology for structure detectors to indicate the structures in distinct directions and scales and then to improve the performance of the proposed method. We will also continue to deeply analyze the method proposed and demonstrate the convergence of the scheme. Some fast numerical schemes for the PDE system could be considered in future work as a result of minimizing the augmented Lagrangian functional. Funding acquisition: Hena Rabbani, Hadia Atta, Nasra Begum.